1. Field of the Invention
The present invention relates to a delay controller, a control method, and a communication system, in particular, a delay controller, a control method, and a communication system which enable images to be switched at high speed.
2. Description of the Related Art
At present, there is widespread use of applications and services that transfer image data (particularly moving image data) via various networks such as the Internet and LAN (Local Area Network). When transmitting and receiving image data via a network, it is common to reduce the amount of data by applying an encoding (compression) process on the transmitting side before sending the data to the network, and reproduce the encoded receive data by applying a decoding (decompression) process on the receiving side.
For example, there is a compression technique called MPEG (Moving Pictures Experts Group) as the most common approach for image compression. When using the MPEG compression technique, an MPEG stream generated by the MPEG compression technology is stored in IP packets that are compliant with IP (Internet Protocol), and delivered via a network. Then, the MPEG stream is received using a communication terminal such as a PC (Personal Computer), a PDA (Personal Digital Assistants), or a mobile phone, and displayed on the screen of each terminal.
Under such circumstances, for applications mainly aimed at delivery of image data, for example, Video on Demand, live video delivery, video conferencing, and videophones, there are environments where not all data on the transmitting side arrives at the receiving side due to network jitter, and environments where image data is received by terminals with different capabilities. It is necessary to take these environments into consideration.
For example, it is possible that image data transmitted from a single transmission source is received by and displayed on a receiving terminal such as a mobile phone having a low-resolution display and a low-throughput CPU. At the same time, it is also possible that such data is received by and displayed on a receiving terminal such as a desktop PC having a high-resolution monitor and a high-performance CPU.
In cases where the fact that the packet receiving condition varies depending on the network connection environment as described above is to be taken into consideration, for example, a technique called layered encoding is used, whereby encoding of data to be transmitted and received is executed in a layered manner. For image data that has been layered encoded, for example, encoded data intended for receiving terminals with high-resolution displays, and encoded data intended for receiving terminals with low-resolution displays are sorted out and retained, and image size or image quality can be changed on the receiving side as appropriate.
Compression/decompression schemes that enable layered encoding include, for example, video streams based on MPEG4 and JPEG2000. In MPEG4, the FGS (Fine Granularity Scalability) technique is scheduled to be adopted and profiled as a standard specification, and it is said that this layered encoding technique enables delivery of data in a scalable manner from low to high bit rates. Also, with JPEG2000 based on a wavelet transform, it is possible to generate packets on the basis of spatial resolution by exploiting the characteristics of a wavelet transform, or generate packets in a layered manner on the basis of image quality. Also, JPEG2000 enables layered data to be stored in file format by the Motion JPEG2000 (Part3) standard that can handle not only still images but also moving images.
Further, as a specific example of data communication that employs layered encoding, data communication based on a discrete cosine transform (DCT) has been proposed. This method executes data communication by applying a DCT process to, for example, image data to be communicated, achieving layering by differentiating between high and low frequencies through the DCT process, and generating packets divided into high-frequency and low-frequency layers.
In many cases, real-timeness is demanded for delivery of such layered-encoded image data. Currently, however, greater priority tends to be given to large-screen, high-quality display than to real-timeness.
To ensure real-time delivery of image data, a UDP (User Datagram Protocol) is typically used as an IP-based communication protocol. Further, an RTP (Real-time Transport Protocol) is used for the layer above the UDP. The format in which data is stored in an RTP packet complies with each individual format that is defined for each application, that is, for each encoding scheme.
As communication networks, communication schemes such as wireless or wired LANs, optical fiber communications, xDSLs, power line communications, or Co-ax are used. While these communication schemes are becoming faster year by year, the image quality of image content transmitted by those schemes is also becoming higher.
For example, the code delay (encoding delay+decoding delay) of a typical system in the MPEG scheme or JPEG2000 scheme which is currently the mainstream is two pictures or more. This makes it hard to say that sufficient real-timeness of image data delivery is ensured.
Accordingly, in these days, image compression schemes are beginning to be proposed which achieve a shorter delay time by splitting a picture into sets of N lines (N is equal to or larger than 1), and encoding an image by each split set referred to as line block at a time (hereinafter referred to as line-based codecs). Advantages of line-based codecs include, in addition to low delay, the ability to achieve high speed processing and reduced hardware scale, because the amount of information to be handled per one unit of image compression is small.
The following are examples of proposals made on line-based codecs. Japanese Unexamined Patent Application Publication No. 2007-311948 describes a communication device that performs a process of appropriately complementing missing data on a line-block basis, with respect to communication data based on a line-based codec. Japanese Unexamined Patent Application Publication No. 2008-28541 describes an information processor designed to achieve reduced delay and higher efficiency of processing when using a line-based codec. Japanese Unexamined Patent Application Publication No. 2008-42222 describes a transmitter that minimizes degradation of image quality by transmitting low-frequency components of image data on which a lined-based wavelet transform has been applied. Since use of line-based codecs has enabled high-image-quality, low-delay transmissions, application of the codecs to camera systems for performing live relay broadcasting is being anticipated. As an example of proposals made for camera systems for performing live relay broadcasting, the present applicant has proposed a system that enhances transmission efficiency by using a digital modulator, as disclosed in Japanese Patent No. 3617087.
Accordingly, as disclosed in Japanese Unexamined Patent Application Publication No. 2009-278545, the present applicant has developed a technique for stably acquiring synchronization in communication using a line-based codec.